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Frnblem statement 

In this paper we discuss the use of structured observations as a research strategy in two recently 
completed studies that investigated students thought processes and behaviors in a microcomputer- 
based laboratory (MBL) environment ami in other instrument-based laboratory environments. 
Structured ol^ervations, as discussed by Krajcik, Simmons & Lunetta (1988)^ require the amultaneous 
videotaping of each observation or treatment session along %vith recording students' verbal commentary 
as they interact with an activity. The videotaped records capture the corrrepondence between 
students' observed actions ai^ their verbal commentary about their predictions, explanations, 
obsm^tions, and procedural decisions. The vidwtape and audio tape records enabled us to compare 
how a student's actions and the instrument readings related to that student's verbalizations. 

Laboratory activities represent complex ta^ environments for students- During latxiratory 
activities, students physically manipulate materials, perform procedures, and make decisions, 
observations, and explanations. TTnis complexity makes a laboratory setting a difficult plao? to study 
learning, but structured observations may be an effective research tool because the researcher can 
simultaneously capture student's thoughts as expre^ed in verbal commentary, the student's actions as a 
result of those ttoughts, and the phy^cal n\anipulations and instrument readings that are part of the 
student's sensory input to the next series of thoughts. 

In the studies reported here, students were encouraged to think aloud as they were videotaped 
performing laboratory activities, and they were sonoetimes asked to clarify and explain their 
expressed thoughts. The videotapes were then analyzed to detCTmine how well the students' actions 
corresponded to their self-reported thought processes. Structured observations recoided in this manner 
provide a rich data set about how students interact both with the instrumentation of the treatment and 
with the cognitive tasks of the treatrrK?nt, 



ThcoreUcal rerapectiye 

Conceptual Framework 

Because this pap^ nepcsts on tlK? appIicati(Hi of the research technique (tf structured obsavations to 
tte study of human cognition, we briefly review tl^ theoretical bases used to support the claim that 
verbal data indicate thought processes whidi give insight into the cognitive structure of the 
individual verbalizing. Three irodds l^p support this claim: 1) a model of cognitive structure is 
i^eded to undo^nd how leame: s oi^nize their knowledge, 2) a model of cognition is needed to 
evaluate how learm^rs" thought process operate during an activity, and 3) a model of concurrent 
verbalization is needed in order to have sorr^ coi^idence that students' v^balizations are indicative of 
their thought processes. Taken together, these n^xiels lend credence to the claim that students' 
cognitive ^ructures and thought processes can be ascertaii.ed to sorrK? degree by interviews and think- 
aloud protocols. 

A HKxlel of cognitive structure is provided by the generative Iraming mcxlel. The generative 
learning model indicates that students leam by building their own cognitive structures (Wittrock, 1986, 
1978, 1974). The primary thesis of this model is that learners generate their own meaning from 
instruction cased on their background, attitudes, abilities, arKl experience. Learners selectively attend 
to the flow of information, and their preconceptions determine the information to which attention is 
paid. The brain actively interprets this information and draws inferences based on its stored 
information. The newly generated meanings are then actively linked back into the learner's prior 
knowledge b«se. In sum, learning is viewed as a cyclical proc^ with new information being compared 
to prior knowk?dge and then being linked back into tlwt kiwwiedge base. 

Osborne and Wiltrock (15«3) extet^ed the nwdel to the acquisition of scientific knowledge, and 
Osborne and Freyberg (1985) presented evidence that ^'children's sdence" plays a mapr role in helping 
or hindering the formation of appropriate concepts and coiKept linkages In C^bome and Wittnxrk's 
elaboration of the generative learning nnodel, students build sei^bk? and col -cnl understandings of 
events and phenonx^na of the world from their point of view. IT^^se understandings can be referred to as 
conceptual structures (Osborne and Wittrock, 1983) or cognitive structures (West, Fensham, and GarranJ, 
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1^). Throughout this paper, we will use tl^ term cx^;nitive structures. Because learners construct 
col^rent uixlCTStai^ings, words such as 'aidd' and liase' are labels f» elaborated c^^Jtive structures 
stored in the brain. West et. al. point out, towevCT, that stwtente' cc^nitive structures for these words 
are often quite different from the cc^;nitive striKiiires of sdenti^ Thrae different cognitive structures 
tl^t students bring to instruction are oiUed alternative fran^works (Driver & Ea^y, 1978). 
Throughout this paper, we refer to studente' cogniti^^ ^ructures Uiat dif^ front tin? views of scientists 
as alternative coiKcptk)ns, Unn et al* (1^) refer to Aese conceptions as intuitive conceptions; 
however, we prefer alternative ccmceptions to intuitive conceptions lm:!ause learners actively construct 
these conceptions by synthesizing their experi^ces, Honce, they app^r to be more than ^'intuitive." 

West, Fensham, and Ganard (1985) have studied student learning in chemistry and have 
attempted to describe the cognitive structures devetopcd by individual learners and by groups of 
learners. They claim that knowledge has two components: public knowledge and private 
understandings. Learning consists of taking the public knowledge presented by instruction and in 
textbooks and relating it to previous knowledge and experience to airive at a private understanding of 
the knowledge that has been presented. They argue that nublic kiK>w}edge is definable but private 
understandings of that knowledge will vary with the individual learner. 

In chenustry. West, Foisham, aiKi Garrard (1985) argue that the two n^in types of public 
knowledge are propositions arMj algorithms They define propwzitions as declarative statements which 
are definitional in nature (An acid has a pH le^ than 7.). Algorithms are defined as public knowledge 
of how to perform a prcKress (neutralize an add), T!^ private understandings which re^lt from the 
learner's acquisition of this public knowledge are a>ncepts and skills. Concepts are a set of propositions 
that a person uS'.'S to infer nr^ning for a particular topic, such as pH, A concept will usually include 
both the public knowlcdj^ definitiortal proportions arwl all other kn jwledge^ public and private, thai 
a person relates to tlK>se prc^TOsitions, A skill is considered to l>e the ability to perform a ^edfied task. 
People who can perform a task are said to possess that skill; they may or may not use the Tjook 
algorithm' to perform the task. 
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This cc^nitive structure c^n be affected by addition (rf new kiK>wtedge bits or tl% creation (rf new 
relationships. Humans cons^tly cluinge thdr oc^nitivc structures (Wrat^ Fen^mnv arrf Garrard, 
1^), and every ol^ervation potentially adds to soira; part oS our cogniti^ structure. T!^ two studies 
discussed in this paper investij^ted students' cognitive structure of add-base chemistry by using 
thought processes to indicate what acid-base concepts were used by students to understand the titration 
activities. 

Hie nK>deI of human cogniticm and of concurrent verbalization used in tlu?se studies was dcvdoped 
by Ericsson and Simon (1984), who argue that verbal data can indicate thought processes. Their model 
views cognition from an infom^tion processing perspective, and tl^ term 'cognition' is applied to all 
stimuli, internal and external, processed by tl^ n^nK>ry. H^^se stimuli arc tln^n sub^ted to o^nitive 
pitxressing, which Ericsson and Smon define as "a sequence of internal states successively transformed 
by a series of information processes'* (f^ge 11). 

The information can be stored in, and acccs^ by, three main menK>ries^ all of which have different 
capacities and access characteristics. First, there are several sensory memories which store 
information for a very short tin^. Second, there is a short-term m^nozy (STM) which has a limited 
storage capacity but can hoW information for a lon^r pericx] of tinw than tlw sensory memories. Third, 
the long-term memory (LTM) has a very large capacity and will hold information relatively 
permanently. However, tlK? LTM has a dow access time and fixation rale (defined below) compared to 
the other m«?mories. 

Within this model, infommtion flows in a definite pattern. A central processor controls the 
information to which an individual will attend and this information is stored in STM- While in STM, 
information is accessible for verbalization and furtter proce^ing. Information can also be transferred 
from STM to LTM for permanent storage. Similarly, information stored in LTM can be transferred 
(retrieved) to STM, where it can be verbalized and/or reprocessed. An important assumption of this 
model is that only information in STM is available for verbal reports. This nieans that students' 
veriwiizalions reflect to sonw dc^jee the contents of STM and provide dues to what information from 
LTM and/or tte environment is being heeded. 
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If stucicnts do indeed learn by generatir^ their own knowledge, then activitieG Ibat they en^ge 
in and their thought jniocesses during the OHirse of an activity beconne important. If, during t le 
activity, students are actively engaged in retrieving corKrepts from long-term menwry, in a:ialyzing the 
activity in terms of tJ^ previous concepts, and in nKxiifying and /or exteruiing the propositions which 
define their scieiKe concepts, thai it is pebble that the information deve' >ped during the activity 
will be integrated in Icmg-term numwry virith information about these concepts gained ^m clas»t>om 
instruction^ text readings, previous laboratory activities, arrd g^^ral knowled^. This integration 
should result in associating more propositions with each concept and in making more associations 
l)etween coiKepts. Thwe two activities would therefore tead to increased comj>relwnaon. These 
outcomes can boA be assessed using the technique of structured obs^aticms because the videotape record 
allows the researcher to compare the students' belwviore with their verbal commentaiy. 
Use of Instnunentatilpn in Sdenw Uaming 

In partial response to concern over students' difficulties in learning science, instrumentation, such as 
microcomputer-based laboratory (MBL) equipment and pH metCTS, might be ustni in the latoratory to 
enable students to doej>en their concepts of such topics as acid -base chemistry. Several studios have 
reported that MBL apparently helps students develop better under5t?»rtdmg of science concepts; 
however, it is important for educational researchers to investigate what students are thinking about 
whon using these instruments to ^rcrtatn if students an^ iiKleed devdofMng delailixi, integrated science 
concepts (Linn, 19S6). 

Unfortunately, we know very little of Vkfhat students, think about when they are engaged in 
laboratory activities, and we know even less about students' observable behaviors and thought 
processes in a microcomputer-based laboratory environrwnt. A number of science education researchers 
have claimed that MBL activities should help students engage in scientific prcKcdures, including 
asking 'TVhat if?" questions (Tinker, 1981) which would indicate that students were actively engaged 
in thinking about the activity. Lunetta, Hofstein, and Giddings (1981) aigue that very little is known 
about the learning outcon^ that occur when students interact with a rognitive task in the environment 
of a bboratory activity. The method of semi-structured observations attempts to investigate the 
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interaction between students* tl^nights aiKl thdr ol^erved actions as theyr perform laboratory activities 
usii^ rracroccHnputa**ba5ed kboi^umes and <Aher fonm erf instnumitation. 

Several leseanihCTS have used quantitatiw iraasuies to examii^ some of the learning outcomes 
which result from u^g MBL in dai^fDon^ (Brasdl, 1^7; Mokrw & Tinker, 1^7; Layman & 
Kirkpatridc, 1988; Lewis & Unn, 1989, Linn & Soi^, 1968; Linn, Layman & Nachnuas, 1987). Several 
studies have explored tht affect of MBL on grai:Mng skills and graph interpretation and have shown a 
positive impact (Mokros k Tinker, 1987; Layman & Kirkpatrick, ^%8; Unn, Layman & Nachmias, 
1987). CHher studies have investigated the affect of MBL on the development of concepts. Three 
studies indicate that MBL can have an influeture on sha{Mi^ conceptual development; however, the 
instruction in which it is en^>edded |rfays a critical role (Lewis & Linn, 1^, Linn & Songer, 1988; Linn, 
et al, 1990; Thornton, 1990). BraseU (1^) investigated in the direct impact of real-time graphing. She 
reported that high school physics students improved in their comprehension of distance and velocity 
graphs using an MBL unit with real-time graphing as opposed to eithCT pendl^ml-paper graphing or 
MBL with delayed graphing. These studies point to potential of MBL becoming a powerful tool in the 
science classroom. To date, studies have not explore ite interrelationships between students' thought 
pocesses and behaviors as they performed the laboratory activities in the MBL environment. 

The present paper describes the use of structured observations in two studies to analyze the 
correlations between the vCTbal commentary and behaviors that ocoir during a think-aloud, activity- 
oriented session. More specifically, the activity focused on the neutralization of adds and bases, using 
either a pH meter, a pH probe interfaced with a microromputer, or the traditional colorimetric 
titration. More qualitative studies are needed to construct models of how students' behaviors when 
performing MBL activities correlate with their thought processes and to how these behaviors and 
thought processes affect their construction of chemical knowledge. 

Use (rf Structured ObsCTvatiims 

Design of the Studies 

In the two studies described bclo%v, students were asked to think aloud as they performed laboratory 
activities using different levels of instrumentation. In c^ch of these studies, students were told that we 
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were interested in what they were thinking as they worked, awi they were instnicted to ^talk aloud" 
and to 'think aloud.'' during tfie activity They were ]»rom{:^ to do so throughout each ses^on. A 
s^iuiard list of prompting comments was used as suggested by Larkin and Rainard (1^). Examples of 
these comments are "Can you tell what you're thinking?'', ''Can you tell me wnat you're doing?'', 
ami Tlcase r^n^mber to think aloud 

SonH!time^ however, prc^nng que^ions had to be a^^ed in onkr to eiKourage students to expr^ 
their thoughts more cleariy and to ascertain what background knowledge they were bringing to bear 
upon the problem. Examples of these probes are: ''What do you think it means to 'neutralisK? the 
acid'?'*, and ''What do you think (the graph} might mean?^ In the first study, tlKse types of probes 
were sometimes asked during the activity, but in the second study these probes were used after the 
activity was completed- 

Design of the First Study . In the first study, students' behaviors and thought processes were 
investigated as they performed an add-base neutralizatk>n. Eight stud^^ts were divided into two 
groups and performed the same tasks but used differwt instrunH!nts. One group used a stand-alone pH 
meter, and the other group used a microcomputer interfered with a pH meter. Students performed the 
activity individually, and thKnr actions and verbal commentary were recorded on */ideotape and were 
subsequently analyzed to correlate tJH?ir behaviors with their verl>alizations. 

In both observation groups the studerits had two task^ to calibrate the instrument using known 
buffer solutions and to rKnitralisce an add. For tte neutralization procedure students were simply 
instructed to ''neutralize the add * All appropriate solutions were labdled and available; however, 
students were not given SjXJdfic directions on how to select and use those materials in order to iwitralize 
the add. It was dedded that m>n^>edfic instructions would nmke greater cognitive demands and 
provide the opportunity for more verbalizations, thereby giving a mwe complete picture of the 
cognibvp proce^es involved. However, specific, written instructions were given for the calibration 
procedure, and all the solutions needed to complete the calibration task were iabdk^. This was done 
because students might not be as familiar with the instrun>ent calibration task as with the 
neutralization task. 
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Stucknts wore a microphora and were vkleotaped as ^cy worked thrcHi^ the mch session, 
jm>viding as full a data set as pos^ble. 71^ vidcota[^ng focused jmirarily wi ctose-up^ of tlu^ir actions 
and on dc^c^ups of the pH n^ter r^ings or of Hhe graphs gen^ated by the com^Hiter wMdi were 
displayed on the computer's nwniton Ths videotapes were tran^nibed, and the resulting protCKX>ls were 
coded using coding categories, as suggested by EricsscHi and Smon (1984), and analyzed by counting 
frequencies in each categcHy. 

pMlgn of thg Second Study, Students' t}K>ught {mx!esses w^e investigated as tl^ perfbnncd a series 
of add-base titrations u^ng eith^ a microcomputer^interfaced pH probe, a stand^alcme pH meter, or a 
cl^mical indicator to detect changes in the pH of tl^ ctonical syst^n« Fifteen students were divided 
among the groups. As the students perfontned the titrations, their verbal commentary was recorded and 
the readings on their instruments were videotaped. Unfortunately, the lighting was not sufficient for 
the video portion of the tape* but the audio portion was used a comparison. 

Students in this study were given an individual orientation session in which they practiced 
thinking aloud and using their instrument. The researclK^r noodellcd tl^ appropriate instrumental 
technique and also modelled thinking abud by giving examples of verbalizations of thoughts (like 
remember that acids are sour^), of (^notions (like ^'I don't understand this, and I feel frustrated''), and of 
thinking processes (like 'TMow I think I'll add some base"'). Then t!^ student practiced using the 
instrument and vert>ali2ing, and the researcher gave feedback when the verl^firing was adequate and 
remiiKled the student to think aloud whcnevCT a pause lasted longer than three or four seconds. If 
verbalizatk)n was inadequate, the student was told that or ste needed to verbalize more and was 
given encouragement to do so. At the end of the titiation, studente were given nriore encouragenient to 
verl>aliz;? every tlwught they had, and students were asked how they felt about doing this activity. 
Data Analysia 

The videotapes and audio tapes of both studies were transcribed, and the resulting protocols were 
coded and analyzed by frcquem^ counting. Coding categoric^ were derived from ihe data. 
Descriptjon Qt the Coding Categories, in the first study, students' transcripts were reviewed and coding 
categories were developed from the i>attems perceived in the veri>alizations* The statements were 

^ i i) 



10 



coded as betonging to one of five categories: procedural statetmmts, analytical statemaits, emotional 
statennnits, staten^ts revi^ii^ an ii^dequate concq>t« ami ^tema^te expressng an adequate concept. 
Some Elements were complex^ and these stateimnts were thCTefore a^grad cod« in two or nnore 
categories. Prequer^ counts of these categories were taken for eadi student. Tat^e 1 ccmtains a 
summary of the coding categories used in this study. Table 2 contains examples that illustrate the 
coding categories. 



Table 1 

Coding Categmigft fnr th^ Analyst of Think- Alnud P»t^ofnU In Sftidy #1 



1. I^tKedural Statements referring to 

a. Rc^ir^ or questioning dinxrtions* 

b. P^Hforming an action. 

c. Stating a goal. 

d. Deciding wh^t to do next or admitting not knowing what to do next. 



2. Analytical statements referring to 

a. C^Tserving^ interpreting^ or explaining events or text. 

b. Undo^iKiingornotumJerstandingobs^ationsortext. 
c- Hypottesizii^ about concq>ts. 

d. Rccallii^ pertirHmt sub^ matter knowledge. 



3. Emotional statement referring to 

a. Puzzlenwnt, frustration^ or satisfaction. 



4« Statements &q>ressix^ in^Sequaie undei^^amlings of 
a. pH, acids, bases, neutralization or buffers. 



5. Statements expressing miequate understvidii^ of 
a. pH, adds, t>ases, neutralization or buffers. 
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Table 2 

Examples the Coding Categories for Stmty f 1 



1. Procedural statements. 

a. T>Iow I'm placing the electrodes in the pH 4." 

'Turn to ^UKlby arei plug in*** 
c. "Are you sure this is ttie right way to calibrate?" 



2. Analytical statements. 

a. Tm wondering how come every time I add one or the other it's always overshooting.** 

b. "As it (tte pH n^rter m^edle) grts hi^ier the line goes up.** 

c. the NaOH, I would think, urn, brin^ all the numbOTS up higher."* 

3. Emotional statements. 

a. Tm conhised right nowr 

b. Tm thinking about how much it would cost to replace this meter if I was io just smash it or 

son^hing." 

c. 'There you arc/* 

4 Inittlequate understandii^ statements, 
a. ''What is a titralionr 

b- *So this would tx? sort of like a peroxide, 1 guess^ siiKe it has hydroger in it." 
c. "Now do the second experiment which is adding pH 7 in the distilled water." 

5. Adequate understanding ^atemenis. 

a. 'Well pH of 7 is like right in the middle, and 1 think that's where the neutralization occurs/' 

b. "....the lower ^de of the scale is for acid and the upper side is for base*./' 

c. The hydrogen chloride, or whatever it is, is add, i^'t it?" 



In the second study, the data were categoric slightly differently because the task demands were 
different. Because shidents performed three titrations of adding a base to three different acids, we 
found it necessary to add an observational ccxiing catc^ry. In addition it was difficult to reliably code 
separate coiKeptual and analytical statements because students would often combine them in one 
sentence, so the two categories were collapsed into one category. The students' statements were 
categorized as procedural statements, oteervational statements, or conceptual /analytical statements. 
Statements were categorized as procedural if they referred to the titration proa?dure, as observational 
if they referred to the changes in the acid-base systems which the students could see, and as 
conceptuaI/anaIyti<^I if the statements referred to the processes of interpreting, analyzing, and 
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predicting in which the ShKients 0\gaged as they paformed titrations. Table 3 summarizes the 
coding eateries used in tl^ second study. Table 4 contains examptes that illustrate the categoric^ 



Tables 

Coding CatggnriM for tiie Analysis of Think^Almid Protocols in Stnc^ #2 



1. Frooeduial statements leforix^ to 

a. Adding milliliters. 

b. Stirring or swirling %hs solution. 

c. Recording the data. 



2. Obsexvatioxial statements refeiring to 

a. Cok>r diang^ of the solution. 

b. pH meter needle nwvcmeit 

c. Vali^ of pH read from the mrtCTiM' microcomputer scr^ 
d* The shape of tftegrafrii on tte compute scr^t. 

e. How fa^ or dow change occurs within the present activity* 

f . Changes in shape, o^Iot, or nH)tion within ti^ pesent activity. 

g. Inability to see anything happening. 

3. Conceptual/Analytical statements relming to 

a. Concepts reciJled from lc»ig-temi n^moiy . 
b* Speculation « to why an event occurs. 

c. Gomparing present observations widi past activities 

d. Predictions as to what might happen i^xt. 

e« Statements that the student is not thinking of anything* 



4« Miscellaneous statement xefening to 

a. Linking words ai^ interjections such as OK, well ,so, wait a minute, 
b* Emotional words such as wow^ whew, crazy^ borings and wild. 

c. Questions arrf/or comments directed to the observer such as look at this^ can I do this without 
goggle^ and did you sec what happened. 
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Table 4 

Examples of thg Coding rAi^ories for Study »2 



1* Procedojral statements, 

a. nVe're gping down to 18." 

b. 'Sur this up aga^n.^ 

c. '^onthekeybcmrdamlentt^.'^ 



2. Observational statements. 

a. 'Seeapnkcc^orwhenyoupour," 

b. nOX)pHunilsat24.'' 

c. ^ecdte sure eiwugh moved up fast that tiim.'' 

d. "Ami it \oc^ like if s goii^ upwaids, if s climbn^ ik>w.'' 

e. 13ecause the pH changes so slowly from when I put in about 22 milliliters it changed very 

f . "Straight line. Doing what it did in the beginning.'^ 

3* Conceptual/analytical statements • 

a. 1 think this acid is stroi^r than tl^ oi^ we used last tin^.'' 

b. "After a certain point the colors weren't like going to change much no matter huw much NaOH 

you add to tte 2^." 

c. "The graph looks diff^irait than last tin^.'' 

d. wonder what happms if >^ just. , , . put N^H in there. . . . and take the pn>t>e and put it 
in there and see if the needle goes up." 

e. "I'm not thinking anything. I don't tWnk that much " 



4« Miscellaneous emotional statements and statements directed to tiie researcher, 
a- "Whoa." 

b. "Itdrovemecrasy yKterday." 

c. iVJow, is this, is this the only thing that going to do?" 



Results and Analysis 

This videotaping/ recording technique a)n^ined with these coding categories for tte verbal data 
allowed us to describe in some detail what student s in an instrumentation environment. For example, 
students in lx>th studies spent a major portion of their lime on pn>cedura] matters and relatively little 
time in observing, drawing conclusions, or ntaking infierences based on ihe data that they were 
accumulating. 



14 



14 

First Stuiiy , In the fii^ Study, the audio tapes enabled us to tabulate the frequencies of statements 
within e^h coding category. Table 5 clearly shows that stu<tent$ \^nt>alized a substantia! nun^bcr of 
fmxxdural staten^ts, and relatively few analytical or coiKeptual statenwnts which could provide 
evidence of their ability to relate the physical manipulations of tlH? activity to their knowledge of the 
subject- 
Tables 

Avgragg Frpqumiy of StatPtngnte Pgr Coding Category by Group for Study #1 



pHMctgOgyp MimKPmptttgGrowp 

Procedural 444* 573 

Analytical 191 73 

Emotional 32 16 
Inad^uate 

im^Titsuuling 11 32 
Adeqv.ate 

underslamSii^ 43 17 



^Number of statements nnaue by each group in each category. 

The videotapes facilitated analysis of students' cs«.&ior^ in conjunction with their verbal 
commentary, and allowed us to identify behaviors which contributed to this involvement with 
procedures. Tte videotape record revealed that four l^haviors were observed in connection with verbal 
conuwntary concerned with procedun?s. 

first, the students ignored what th^ didn't understand* For examphv some students skipped the 
instruction lo check the temperature setting on the pH meter because they couldn't find the temperature 
knob. One student said '^I'm not sure where the temperature dial is, but uh Til just have to skip.'' 

Seconds students forgot several steps in the written procedur For example, students would 
prepare a beaker of solution but fo. get to place the pH meter electrode in the solution* One student in 
Ite pH meter group faikxl to adjust the pH setting on tl^ n^ter despite explicit instructions to do so. 
Two other students correctly adjusted the nwter Hiding; another student ^nt considerable time 
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attempting to adjust the meter before realising that the meter was set to standby and could rK>t respond. 
These two behaviors would occur often at the beginning of the activity, but they would also occur 
thn>u^out the activity. 

Third, if a ^ludent became confiased, the geno'al tendeiKy was to stop and start all over again, and 
fourth, some students were unable to recognise when a task had been completed. For example, several 
students finished both tasks and then started over again because they did not recognize tJ^t they were 
finished* In tl^ microcomputer group, students had particular difficulty in recognizing whon the 
calibiation ta^ was completed. Each student would fmish the calibration, hesitate, reread the 
directions, and then decide to go back and redo the calibration. One student performed the calibration, 
didn't realize that the task was completed, and performed the calibration again. Later in the 
observation the student went back and calibrated a thinJ time. Two students performed the calibration 
incorrectly the first time, did the calibration correctly the second time, and started to neutralize the 
add. They could not decide on a gc»l for the neutralization so they repeated the calibration for a third 
time. When asked why they were doing the procedure, they replied ''to calibrate the prd>e'', "to get 
more data.*" 

The videotapes also revealed several instances when students in the microcomputer group failed to 
see the pH nreter interfaced to the microcom|niter as a ronnected whole. One student calibrated the pH 
probe without realizing that the computer program was on the wrong selection. The same student, 
making a second attempt at calibration, ty^r HI in the correct pH values on the computer but left the 
electrode lying on the counter top. This student also nr^de m> attempt to adjust the pH meter reading; 
he/she simply accepted whatever value was displayed* 

Second Study. In tl^ secorKi study, the audio portion of the tape wao used to code the verl^l 
commenlaiy and to compare the sequences of behaviors acro» treatment groups. This methodology 
allowed us to analyze how each group interacted with a specific technology. 

Frequencies across Crnnpg. After coding the think-aloud protocols, the frequencies of each category 
v^rerc romputed and group av*:rages were compared across treatn^mt sessions and acn^ groups. The 
frequency of p\ jcedural statements decreased within a group across treatment sessions. This was 

If; 
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expected because we had predicted that the number of proc^ural statements would decrease as the 
^udcnts bc<^nw nK)re familiar with t!^ routine of titration. We had also predicted that tl^ frequeiKy 
of analytical staten^ts would increase as students becan^ im>re familiar with the titration routine, 
but we found that th^ frequemy of conceptual /analytical staten^nts^ as well as observational 
stat<Hnents, decreased for the secoml treatment ses^on emd then rose in tlw third trealn^nt session. 
This increase was possibly due to the use of phosphoric add in that session berause phosphoric acid 
ioni2^ in several stages and therefore exhibits a slightly different behavior than the previous two 
acids. 

Across groups, the microcomputCT group consistently had tin; highest frequency of procedural 
statements^ which might be because the microcomputer group students were rcquirei to enter the 
milliliters added into the computer and the other groups were required to write down the number of 
milliliters added and their observations of oolor changes (chemical indicator group) or of pH values 
(pH meter group). It is possible that the act of entering data was sufficiently different from writing 
that the microcomputer students verbali^^d this step arKi the other students did not. For example, in 
treatment sequence 2 the following sequence occure (#0301 .JC page 2, the underlined statements are 
procedural): ^ 

Swir! that around. 
It is going to 4J1 

Compare that sequence to one from the chemical indicator group (#0110.DH, page 2, tK underlined 
statements are procedural): 

Stir this up again. 

OK. 

Bubbles. 

This is getting boring. 
Qear^ see son^ bulimies. 
1^ 



^The first two digits df d» studem's number give tte trcatmoi^ i^nnip, aiKl the last two digits are sequential for 
(nteing puipo^^ The tettm serve to further s^tify^disid?}^ Ther&f<^ this stiKk»it is in tl^ third treatment 
group is the fira ^uctent in the whote ^mple. 

ERIC , ^ 



17 



There, 

Also ccmipare tl^se sequ^iKes to one from Hhe pH n^ter group (fOE^.DR, page 3, the underlined 
staten^ts are prcK^edunil): 



And I am going to stir this again. 

See sonwthing changing. 

Oh, hunv it is moving up to 4.1 pH unit now. 

Wonder why it won't ^art at 1 or a 0 you know. 

The needle, why does it start at 4.00? 

Hum, I don't know why. 

These three sequences all involved stirring, making <^»ervations and infereiKes, and then adding 
more NaOH. Ffowever, only tt^ miciocom{mter group verbali2ed the act of recording the data by 
saying the word Return.'' Again, prcs^ng ^tum"" on the computer is not as automatic as writing down 
the pH. 

The microcomputer group also had tte lowest frequency of coiKeptual /analytical statements. 
However, it is possible that these stateimmts, although fewer in number are more meaningful. 
Consider the following sequence from the nuooccnnputer group (#Q3Q2.JC, treatnxmt sequerux? 3, page 2, 
conceptual/analytical sequemres are underlined): 



22. 

Hum. . . . 

It is different from other graphs we did. 
This is g^r^ up 

An increase in the pH a lot slower than the other graphs we did. 

Lef s tum this toward me. 

That's 24. 

Make sure. . . . 

This is totally . . . . 

Now at 26. 

Almost turned to water. 

Mavbe by the Hme the experiment is done I co uM probably drink this 

Stjjff. 

28. 

This sequence indicates that the student compares ihe graph being fonned with the graphs 
produced in previous activities. This comparison is possible because of the real-time capability of 
MBL Also, notice that the student makes a prediction as to the probabte nature of the products of the 
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r^ctton. Compare that sequeice to Une ibUowii^ one from the pH nveter group <#(£209.DR, treatment 
scquoKe 3, page 4, amcefHual/analytical sequeiKss are uiuierlined}: 



It ^ keep gra^ cm down a^n. 
VmfAL wonder what happened there. 
Ihad2pointa 

Cm€98 thfe Stilt phpsphprfe ac<<^ tf^at's in frsre- 

SHIL I think thafs stifl nwre phosphoric add in here now. 

lust 2 milhlitere won^t do any good 

So I guess I got to tet it go 2 nvHie do wn r»> w 

To four or something. 

This sequence mainly involves an assessn^t of the condition of the plrosphoric add and a dc?ciaon 
to proceed with adding tlw NaOH. NaOH is the base used in all ti^ titrations, and the student is 
judging whether the phos]^ric add has been completely titrated or whether nwre base should be 
added. Tt^ whole sequence is more tentative than the microcomputer sequerKre. 

A comparison of these two sc^imoits irKiicate that students in the MBL environnwnt tended to focus 
their commentary on more conceptual issues. Apparently, the graphic nature of MBL and the real-time 
graphing capability fostered this level of conceptualization. 

The pH meter group consistmtly had the higher frequency of observational statements. This 
might be because the students focused on two phenomena: the movement of the needle and the pH 
value on the scale. For example, con^der tlw following sequc»Ke from tlw pH meter group (#0207.CS, 
treatment sequence 3, page 2, observational statements arc underlined): 

Acklsomemoie. 

l\ }9 gmg my <^9m on m W - 

At 42. 

Hum, . . . 10.7 . 
44. 

Comes up fast 

Sometimes it is stow and other times . . . . 

Looks like it is poing to stay at 11. 

46. 

2 more. 

It moved a slight bit to 11.2. 
At 48. 

2 more for our la^ two. 
The last reading is 11.4. 
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These observationa] stab^mmts r^rt both the rra)vcn^t of the pH imrtcr needle and the pH 
values on the scale to which the needle points. Tl^refbre it is not surprising that this group had a 
hi^K^* average frequency of ob^rvational data. TaMe 5 presents a breakdown of tl^ frequencies of 
sbitenumts by treatment session and by category. 
Table 6 

Avig-agg Frequency erf Statgmenta Pgr Cndlng Cafpgmy by T»atmgnt S«sim% and by Grmip for Study #2 

farocedmi QbsenratiQnid Conceptual/ Analytical 

Treatment 1 2 3 1 2 3 1 2 3 

Chemical Indicator Group 

40 29 23 41 28 ^ ^ 19 14 

pH Meter Group 

42 34 34 74 58 62 48 26 40 

MScrocompiiter Gitmp 

53 42 36 44 26 33 20 10 15 



Pattern of verbal commentary. The protocols were aralyzed by examining the sequences in which 
the coded categories occurred during the think aloud* No pattern was detected when each coded 
statement was counted separately l7ecau% a pattern of pix>ccdureol>servdtionKX)nceptua] would then be 
different from a pattern of procedure^procedure-ot^ervation-a>7Keptual although the two patterns 
would actually be very similar. Therefore only the sequencing of categories was followed, and no 
attmpt was made to count specific numoer of statements in each aitegory* Of the 42 patterns that 
were analyzed, 43% of them were the sequence fprocedureH^bservationln-conceptual/analyticah In this 
sequence tl^ procedure-ot^ervation segrrNmt might be repeated n times before being followed by a 
conceptual/analytical segment. Over the entire sample of students, 29% used this sequence for every 
treatment session. 
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Fogus of studgnfa ob^rvation- The Structured ob^rvation tcchniqi^ was most hdj^l in 
detemining tfie focus of student's oteenations. Ead\ protocol was amiyzcd to detormiiM? what was 
tte focus of tte student's oteervations during each erf tiie three titrations. The fc^s was detemnined by 
determining tlw ob^ of each observatk>naI ^tenwnt. A <kfinite and different pattern developed for 
each treatment group. For the fir^ two titrations all the stiidents in chemical indicator group 
focused on color, Init sonn^ of ihmx also focused on bubbles^ shadows in the beaker, ami characti^i^ics of 
the liquid, sudi as "oilincss.*' In the third titration, all except one member of the group focused only on 
color. 

Tl^ following sequences from a student in the chemical indicator group illustrate this shift in focus 
(#0103.CP, treatment 2, pages 2-3): 

Four. 

It doesn't sn^ll that good. 
Six* 

Trying to look for the bubbles and all of that stuff. 

Six milliliters. 

Clear 

Still clear. 

Looks the same, 

10. 

Still clear, nothing at all. 
12. 

R: What are you thinking? 

We are ^tting bullies inside the solution. 

You know, like we h^ the bubbles last tin^. 

14. Think I saw a little pink change. 

There, I guess between 12 and 14 is when the solution begins to 

react. 

16. A little bit n\ore pink. 
Clear^ 

And seems solution turns bright pink to clear. 



In the third treatn^nt sequence this san^ student focused exclusively on the color (#0103.CP, 
treatment 3, page 3): 

Rnir. 

It is still the same. 
No chan^. 



21 



21 



Four to it's up to six. 

Shucks. 

OK, no d)ange« 

OK^ adding two more. 

OK, I don'^t see any chai^ in it. 

All right. 

It really is ... , 

SiiKe we're usir^ lesser anK)unt of add, the color change slwuld 

beabmit 

lOIgiK^ 

Close to 10 1 ihink. 

Eight milliliters. 

No change. 

OK, 10 now. 



R: What are you thinking? 



I think tl^ color was light red, but it is more like an orangish cotor. 
You think it is? 
OK. two milliliters* 
Gosh. 

So, I guess the color was light orange. 



For the first two titrations in the pH n^ter group, all students focused on tte movement of the pH 
me^ needle and/or on the pH value, Init two students also focused on the thickness of the solution, 
bubbles, and the tip of the pH probe. By the third titration, all members of the group focused on the 
movement erf the pH meter needle and /or the pH value. Most of the students focused on both the pH 
meter raedle and on the pH values to whidi tl^ needle was pointing. 

Tl^ following sequences from a shident in the pH n^ter group illustrate this shift in focus. Firet, 
tlw student focuses on bubble^ needle movement, ami pH values (#0207.CS, treatment 2, f^ges 2-3): 

Down two more. 
Ami now it is six. 
n^rroter is43. 

I think this add is stronger than the one we used last time. 

Add son^ more. 

Going down to eight 

SonvHthing is buttling at the bottcm. 

It moved to 43 

Going to ^Id some n^re. 

And it move to 45. 

Now up to 4^, that is 12. 
Down2mc»ie. 
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That is 5-1. 
14. 

Two more. 
That is 5,1. 
14. 

TwomcH^ 

Wonder if you shake it th^ bulges will go away? 



By the third treatment sequence, the student focuses only on the pH nxrter needle and the pH values 
(#0207.CS, page 2): 



Down to 18. 
2 point 4. 
20. 

Vm ^1 trying to go oi^ at a time. 
Looks like Vm goii^ to get to 55 
TwonK»e* 
Down 21 

It is jumping back and forth on me a littte bit. 

It is at 6.1. 

22. 

Down two more. 
24. 

We got 6.5. 
Add two more. 
63, oh no, 6.6. 
Add two more. 
6.8. 



The microcomputer group fcxrused cm tlu? shape of the graj* in the three titrations. Two of the 
students in the group did not vei1>aiize eiK>ugh observations to ascertein their focus. Two students 
followed the vertical rise of the graph very closely, paying particular attention to the fact that on a 
color monitor these vertical lines are multicolored. In the first and second titrations, pH values, which 
were also displayed on the screen, were also a focus of observation for a student. 

The following sequence illustrates the typical focus of a microa)mputer group titt^tson {#0312.NM, 
treatment sequence 3, page 3): 

Swirl it around. 
24 into the computer. 
Return. 

It's got an S shape. 
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More com|:^ S than tN? other ones. 

S^ms shape 

but iHOTC cranpKt* 

And t}K?re is m multt-cotonx) line. 

• • • » 

But there is 

• • • • 

tins gpes aInK>st straight up. 
And it is multicolored. 
That could t>e it, 
Point where it is neutral. 
pH 637. 

26 into the computer. 
Return. 

Still continue into an S shape. r 
28rK>w. 
Right there. 
Swirl it around. 

pH gpes up and tto) it comes t>ack down. 

Enter 2S into t)^ computer. 

Return. 

Still continues a S with a slight upward swing. 

The observational sta tenants were very helpful in determining the focus of the student's 
obscrvaticMis. Ericsson arnl Simon's modd of cogitation states that vert>al rqports of observations indicate 
what information is being attended to and processed in short-term memory. Therefore the 
observational staten^nts were an index of what phenomena were txnng attended to by the student. 

The chemical indicator group son^times attended to irrelevant phenonnma. SonietinM^ the 
students sccnwd not to know what to observe^ so tl^ observed everything. TlK?se 5itudents also had no 
immediate, visual method of evaluating the progress of tlw titration. There were periods of time when 
nothing apparently happened because the color change did not come until neutralization, and thcrefoa* 
some of tlw students decided tJ^t no reaction occurred until the color change cMrurred. There were other 
periods in which the color came and faded a^in. Orw or two students did guess that this was due to a 
localized concentration of base because ti^y iK)ticed that the color faded upon stirring. 

The pH meter group had a more difficult problem with respect to their use of short-term memory. 
These students were aware of two things: the moverwnt of the meter needle and the pH value. 
Therefore their observational statements indicate that they were mentally juggling three variables: 
pH value, needle moven^nt, and the number of milliliters of tose added. This burden may have 
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exceeded their nwirory ca^^ty, and prevented ttem from speculatii^ more about the reasons this 
behavior. 

The microcompuler group on the other hand had a relatively low demiand placed upon their short- 
term memory because tl^ microccmiputw disp^yed the graph as it was being fom^. Sometimes 
studcnte %vouId delay entering tl^ cteta into the computer in order to predict wl^ther the lir^ would go 
up or not After entering the data the ^tidents vmuld then study the graph and decide if it was going to 
resemble the graphs of other £Kids. In a real sense, a>mputer se^ns to be functioning as an auxiliary 
nKmK>iy (Unn & Songer, \%8\ displayii^ a visual recOTd of tl^ titration for tl^ student to reference at 
any time. 

The microcomputer group also h^ a very narrow focus. Tl^ students focused almost entirely on the 
shape of the gf^pK particularly the n^on where tl^ graph begins to rise steeply. According to 
Ericsson and Sintt>n's nKxiel of cognition, this focus on only one piece of information, i.e. the shape of the 
graphs probably did m>t represent a cognitive overioad for the students' short-term memory. Therefore 
more of their short-term nr^mnory was available for information proce^ing and retrieval. 

The microcomputer group had availal>le three kinds of information: the volunte of base added, the 
pH value of the solution after each addition of base, and the on-screen graph which was constantly 
formed during tl^ titration. This is Bn amount of informdtk>n »milar to the dnfK)unt of information 
possessed by the pH group; towever, the microcomputer group'?. r.Uor^ nation was not transient. 
Therefore, tlw microcomputer students did not need to attimpt to su re in mmnoiy any of the information 
to which ilwy were attending. All of the information collected in the titration was displayed in the 
graph at all times. 

It is reasonable to speculate that the on-screen graf^ enabled the microcomputer students to focus 
their thoughts on what was happening and why it was happening ratlnr than trying to remember 
what had happened and i^multaneously trying to think about why it had happened. The nnanner in 
which the computer presented the information po^bly allowed the computer to function as an 
auxiliary nwmory device, as su^ested by Linn & Songer (1988). By maintaining the graph as a constant 
reference for the student, the computer allowed the student to use his or her short-term rm?mory to make 
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predictions or cxmstnict possible explanations for tin? graph. Th^fore, ^drats would have been more 
actively engaged in acce^ng their Icmg term memory^ ami cognitive testrucluring should have occurred 
which ^ould have led to a more detailed understanding of the titration. 

Educatlflnal Significance of the Methodoli^ 

The videotape and audiotape record enabled us to examine how well the students' verbal 
comn^rntary agreed with their actual l>ehaviors while uang MBL or otiw technologies. At times 
students' cormrcnts were found to be incongruent with tteir actions^ and at other timi^ rtudents' verbal 
commentary enabled us to ascertain the focus of their attention while they were performing the 
physical manipulations of the laboratory. The videotapes and audiotapes also enabled us to examine 
students' ]K)ught processes, as indicated by tiieir vcrl^a] commentaiy, in ordw to provide some insight 
into the different types of thoughts whidi were heeded in short-term memory. 

The coding catc^PTS proved to be a powerful method of analyzing verbal data for patterns. In the 
first study, these coding categories allowed us to identify a high percentaj^ of procedural 
verbalizations, and the videotape enabled us to identify four l^haviors which may have contributed to 
the students' high frequOTcy of procedural verl>alizations. Without tin? videotape record of these 
sessions, such a correspondence would have been difficult enough to uncover, let alone substantiate with 
convincing evidence. In the second study, the coding categories allowed us to examine the sequence and 
patterning of the verbalizations to uncover a common pattern, Iproccdure-observationln- 
conceptual /analytical, used in nearly half of the protocols. Again, this correspondence might not have 
been observed without this methodology. 

In summaiy, videotaping and audiotaping thi^'^-k-aloud treatments and developing coding categories 
for the vertel data seem to be a powerful way of analyzing the interactions l>elween students' thought 
pnxxrsses and their observable actions. This technique allowed us to gain tentative evidence for why 
MBL may l>e a powerful instructioi^l tooL The verbal commentary of students using the MBL indicates 
that these students atteiuied to features that were more conceptually demanding. The real-time 
graphing ca];^bility of MBL served as an auxiliary memory device for students. 
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Structural obsCTvation may also play a central role in d^gning instruction. Using tiie technique, wc 
can modify the iratnicti<»ial compoi^t, and observe in students pattern of veibal comn^taiy and 
behavior is modified toward conceptually demanding activities. 
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